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Abstract 


We  consider  various  formulations  of  the  notion  of 
descriptive  complexity.   We  obtain  results  relating  dif- 
ferent proposed  definitions  of  an  infinite  random  binary 
string.   We  prove  that  the  set  of  optimal  programs  is 
immune  and  as  a  consequence  obtain  that  the  descriptive 
complexity  of  a  program  given  what  it  computes  is  depen- 
dent on  which  formulation  of  descriptive  complexity  one 
is  considering.   We  prove  that  in  a  certain  strict  sense 
for  one  formulation  of  descriptive  complexity  there  is 
no  optimal  universal  computer.   We  then  develop  the 
notion  of  descriptive  complexity  for  a  subrecursive 
formalism. 
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INTRODUCTION 

This  thesis  will  consider  an  assortment  of  problems 
in  the  area  of  descriptive  complexity  theory.   We  view  a 
computer  as  an  algorithm  for  a  partial  recursive  function 
which  reads  a  binary  string  input  and  then  may  or  may  not 
print  another  binary  string  which  is  its  output.   Relative 
to  a  particular  computer   C,   the  descriptive  complexity 
of  a  particular  string   s   is  defined  as  the  length  of  the 
shortest  string   p  which  when  inputted  to   C  will  cause 
C   to  output   s.   This  notion  has  been  formalized  in  a 
number  of  different  ways.   We  consider  three: 

1.  Kolmogorov  and  Chai tin's  absolute  complexity  in 
which  C  with  input   p  must  output   s   given  no  addi- 
tional information. 

2.  Kolmogorov  and  Chai tin's  conditional  complexity 
in  which  C  must  output   s   given   p   and  information 
about  the  length  of   s. 

3.  Chaitin's  recent  definition  in  which   C  must 
output   s   given  a  program  p  which  is  self -de limiting. 

Though  Chaitin  and  Kolmogorov  were  co-founders  of 
the  notions  of  descriptive  complexity  as  outlined  in  1. 
and  2.  above  we  will,  for  expository  purposes,  refer 
only  to  Kolmogorov  in  order  to  distinguish  these  earlier 
formulations  from  Chaitin's  recent  formulation. 
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We  also  consider  the  relative  complexity  of   s 
given   t  by  which  we  mean  the  length  of  the  shortest 
program  p  which  will  cause   C   to  output   s   given  in- 
formation about   t.   Different  formalisms  arise  from 
considering  various  different  ways  in  which   t   can  be 
given,  i.e.  either  by  assuming   t   itself  is  given  or 
by  assuming  a  string   t'   from  which   t   can  be  computed 
is  given. 

In  terms  of  each  of  the  three  above  approaches  a 
random  or  patternless  string   s   is  defined  as  one  whose 
shortest  program  is  of  length  essentially  equal  to  it- 
self.  Intuitively,  if  a  string  contains  some  pattern 
then  that  pattern  can  be  exploited  by  the  computer  in 
outputting  the  string.   However,  if  a  string   s   is 
genuinely  patternless  the  shortest  program  for  outputting 
it  might  consist  of  inputting   s   and  instructing  the 
computer  to  output  its  input. 

Chapter  1.  will  present  most  of  the  fundamental 
notations,  definitions  and  results  developed  for  the  most 
part  by  others  which  are  relevant  to  our  work. 

Chapter  2.  will  consider  five  definitions  of  the 
notion  of  infinite  random  sequence.   Two  have  been  pre- 
viously proposed  and  subsequently  proven  equivalent.   Two 
have  been  proposed  by  Chaitin.   We  show  that  one  of  these 
is  at  least  as  restrictive  as  the  other.   We  then  propose 
a  new  definition  based  on  the  Chaitin  definition  of  de- 
scriptive complexity  which  we  prove  is  at  least  as  re- 
strictive as  the  first  two. 
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Chapter  3.  considers  the  set  of  optimal  (shortest) 
programs  proving  that  this  set  is  immune.   We  then  con- 
sider the  relative  complexity  of  an  optimal  program  given 
what  it  computes.   We  prove  that  with  respect  to  this 
question  Chaitin's  definition  of  complexity  differs 
sharply  from  Kolmogorov' s . 

Chapter  4.  deals  entirely  with  a  result  based  on 
Chaitin's  formulation:   we  prove  that  given  any  optimal 
universal  computer  there  exists  another  optimal  universal 
computer  for  which  the  complexity  of  all  sufficiently 
long  binary  strings  is  uniformly  reduced. 

Chapter  5.  deals  with  various  definitions  of  de- 
scriptive complexity  where  the  function  defined  by  the 
computer  C   is  assumed  subrecursive .    Various  results 
are  obtained  and  comparisons  with  the  previous  work  are 
made.   The  reader  should  note  that  in  Chapter  5.  we  will 
state  and  use  many  results  concerning  our  new  sub- 
recursive  formulations  without  proof  since  it  should  be 
evident  that  the  proofs  of  these  results  follow  the  same 
lines  of  reasoning  used  to  prove  corresponding  results 
earlier  obtained  for  the  full  recursive  formulations. 
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Chapter  1 
FUNDAMENTAL  NOTATIONS,  DEFINITIONS  AND 
RESULTS  OF  DESCRIPTIVE  COMPLEXITY 

1.1.   Notational  Conventions 

Let   X   =  {0,1} 

X*  =  {strings  of  finite  length  over   X} 

X°°  =    {infinite  strings  over   X} 

A   =  empty  string 

N  =  {set  of  natural  numbers}. 

00 

u,v,w,x,y,z  will  denote  members  of  X  . 
p,q,r,s,t  will  denote  members  of  X*. 
i»  j /k,  j!,,m,n  will  denote  members  of   N. 

oo 

a,B,Y        will  denote  members  of   X*  L^  X  . 

00 

If   P(n)   is  a  predicate  then  9riP(n)   will  mean 
that  there  exist  infinitely  many  n   for  which   P(n)   is 
true.   For  any  string   s  e  X*,   |s|   will  denote  its 

length  (e.g.   |A|  =  0;  |0100|  -  4). 

oo    n 
For  any  string   a  e  X*  U  X  ,  a   will  denote  the 

initial  segment  of   a   of  length   n.   If   n  >  Ia|   then 

a  =  a.   Also   a   will  denote  the   nth  bit  of  a. 

X*   is  assumed  ordered  in  the  conventional  way; 
that  is,   s  <  t   if  cither   |s|<|t|,   or   |s|=|t| 
and   s   precedes   t   in  the  lexocographic  ordering  with 
0  <  1.   We  associate  the  integer   n  with  the   nth   ele- 
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ment  of   X*.   By   n   we  will  mean  the  length  of  the 

string  associated  with   n. 

If   s  =  s,s-...s    then  by  ¥  we  denote  the  string 
12    n 

s,S-S„s_...s  s  01.   By   <s,t>   we  denote  the  string   st 
112  2     n  n 

from  which  both   s   and   t   can  be  unambiguously  re- 
covered.  We  also  denote  by   <n,t>   the  string  qt 
where   q   is  the  string  associated  with   n.   Obviously 
I <s,t> I  =  2|s[  +  |t|  +  2. 

oo 

when  referring  to  a  measure  on   X   we  mean  the 
usual  product  measure  on   X   relative  to  the  probabil- 
ities  •=-  for   0   and   1. 

1.2.   Fundamental  Definitions  and  Results  of  the 
Kolmogorov   Formulations 

We  recall  that  by  a  "computer"   A  we  mean  any 
algorithm  for  a  partial  recursive  function. 

Def.  1.2.1.   (Absolute  Descriptive  Complexity) 

Let   A   be  a  computer  which  maps   X*  ->■  X*   then: 


If  one  chooses  to  think  of  A  as  a  stored  program  com- 
puter then   p   should  be  thought  of  as  program  +  data. 
If  one  thinks  of   A   as  a  particular  Turing  machine  then 
p   is  the  initial  tape  content. 
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It  has  been  established   that  there  exist  universal 
machines   U  which  are  optimal  in  the  following  sense: 


Vs[K^(s)  <  K^(s)  +  c^^y]. 

In  other  words  the  complexity  of  any  string   s   is  at 
most   c     bits  longer  measured  relative  to   U   than 

measured  relative  to   A  where  the  constant   c     is 

independent  of  the  particular  string   s   and  depends 
only  on  A  and   U. 

Given  A„,  A,,  ...,  A  ,  ...   some  particular  ef- 
0   1       n 

fective  listing  of  all  "computers,"  then  relative  to 
this  particular  listing  we  can  diifine  a  universal  com- 
puter  U   by  the  requirement: 


Vp[U{o^lp)  =  A^(p)] 


then  clearly: 


Vs[K  (s)  <  K   (s)  +  (i+1)] 
u        A^ 


If      U        and     U        are   two   universal   computers   then: 
12 


Vs[K„    (s)     <    K„    (s)    +    Cn  &    K„    (s)     <    K„    (s)     +    c^] 
"1-^2  -^  ^2  ~      ^1 


and  hence ; 


V3[|Ky    (s)    -   Ky    (s)  I     <_  c]      where      c    =  max{c-|^,C2} 
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We  can  therefore  choose  a  particular  universal 
computer   U   and  define   K(s)  =  K  (s) ,   and  then  read 

all  of  our  results  as  accurate  to  within  a  constant. 

Since  there  exists  a  particular  computer  A'   (the 
identity  computer)  which  outputs  its  input  we  have: 


K^. (s)  =  Isl 


and  since 


Ky{s)  <  K^, (s)  +  c 


we  conclude  that 


K(s)  <_  I  si  +  c. 

By   K(n)   we  mean   K(t)   where   t   is  the  string 
associated  with   n. 

Intuitively  a  string   s   is  random  if   K(s)   is  ap- 
proximately  |s|.   We  now  define  the  notion  of  randomness 
for  a  finite  string  in  terms  of  a  parameter   c   which  can 
be  chosen  arbitrarily  as  a  positive  integer. 

Def.  1.2.2.   A  string   s   is  random  if   K(s)  >  |s|  -  c. 

4 
THEOREM  1.2.1.    The  number  of  strings  of  length   n 

which  are  random  is  at  least   (1-2     )2  . 

Proof:   By  a  simple  counting  argument  there  are   2 
strings  of  length   n   and  only   2       programs  of 
length  less  than  or  equal  to   n-c.   Hence  at  least 
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(2^  -  2~^      )      strings  of  length  n   are  random. 

Q.E.D. 

If  K(s)  =  n  then  there  exists  at  least  one  pro- 
gram p  such  that  IpI  =  n  and  U(p)  =  s.  We  denote 
the  set  of  such  programs  by  S*.  S*  has  at  least  one 
element  but  may  have  more. 

Def.  1.2.3.   s*  =  min{t|t  e  S*}.   (Here  min  refers  to 
our  assumed  ordering  of   X*.) 

Def.  1.2.4.   s**  =  min{t|t  £  S*   and  for  all   r  e  S* 
the  number  of  steps   in  the  computation  of   U(t)   is 
less  than  or  equal  to  number  of  steps   in  the  computa- 
tion of  U{r) .} 

Both   s  -•■  s*   and   s  -*-  s**   associate  a  unique 
program  with  the  string   s. 

We  now  define  the  relative  complexity  of   s   given 
t.   Here  we  assume  that  the  underlying  computer  is  an 
algorithm  for  any  partial  recursive  function  of  two 
arguments  mapping   X*  x  X*   into   X*. 

Def.  1.2.5. 

(min  IpI 
K^(s/t)  =  U(p,t)  =  s 

(.<»  if  Vp(A(p,t)  /   s). 

g 
As  before  it  has  been  shown   that  there  exists  an  optimal 

universal  computer   U   for  which: 

Vs^tLKyCs/t)    £  K^(s/t)    +   c^^u^- 


Also  if   U   and  U   are  two  universal  computers  then; 
VsVt[|K^^(s/t)  -  Ky^(s/t)|  <  S^,U2^- 


We  then  pick  a  particular  universal  computer   U   and 
define: 

K(s/t)  =  Ky(s/t)  . 

We  also  have  as  before: 

K{s/t)  <_  Is|  +  c. 

Without  loss  of  generality  we  will  assume  a  single 
universal  computer   U  which  is  universal  for  the  ab- 
solute and  relative  definitions  of  complexity  simultan- 
eously. 

Of  particular  importance  is  what  Kolmogorov  calls 
the  conditional  complexity  of   s:  K(s/Is|).   [Note  that 
U(t,n)   is  defined  as   U(t,n')   where   n'   is  the  binary 
string  associated  with  n] . 

Def.  1.2.6.   A  string  is  conditionally  random  if 
K(s/|s|)  >  |s|  -  c. 

By  the  same  counting  argument  used  to  prove 
Theorem  1.2.1  we  have : 

THEOREM  1.2.2.*^   The  number  of  strings  of  length   n 

which  are  conditionally  random  is  at  least   (1-2     )2  , 
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THEOREM  1.2.3.2  K(s/|s|)  <_  K(s)  +  c. 

Proof:   Consider  a  computer   C   which  given  inputs   q,r 
satisfies : 

C(q,r)  =  U(q)  . 

Then   K^(s/|s|)  =  K(s)   and  since   K(s/|s|)  £  K  (s/jsl)  +  c 
we  conclude: 

K(s/|s|  )  <_  K(s)  +  c. 

Q.E.D. 

1.3.   Chaitin's  Formulation  of  Descriptive  Complexity 

9 

Chaitin  has  proposed  a  different  approach  to  the 

definition  of  descriptive  complexity.   His  approach  is 
best  explained  by  considering  a  Turing  model  of  computa- 
tion in  which  there  are  two  tapes.   The  computation  be- 
gins with  the  program  p   inscribed  on  the  input  tape 
and  the  reading  head  scanning  the  first  square  of   p. 
The  work  tape  is  initially  blank  if  one  is  dealing  with 
the  absolute  complexity  and  contains  information  about 
t   if  the  computation  is  relative  to   t.   The  output  is 
found  on  the  work  tape. 

We  now  give  a  concrete  definition  of  descriptive 
complexity  couched  in  terms  of  a  Turing  model  of  computa- 
tion.  [An  equivalent  abstract  definition  which  is  not 
restricted  to  only  a  Turing  model  is  given  by  Chaitin   . ] 
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Def.  1.3.1.   Let  A  be  a  particular  Turing  machine 

/min  p 

H  (s)  =  j hip)    =   s   and  when  A  has  halted  it 
y  is   scanning  the  last  square  of  the 
1    input  with  the  reading  head  never 
I  having  scanned  a  blank  square. 

\j»     if  Vp(A(p)  7^   s) 

Loosely  stated,  Chaitin's  definition  requires  that  any 
program  p  be  self-delimiting.   The  reasonableness  of 
this  definition  derives  from  the  following  two  considera- 
tions.  First,  it  can  be  argued  that  the  Kolmogorov  de- 
finition of  complexity  allows  for  a  hidden  factor  which 
uniformly  decreases  the  complexity  of  any  sequence.   That 
is,  when   U   computes   s   from  p   it  ipso  facto  has  a- 
vailable  to  it  information  not  contained  in  p   since  it 
is  able  to  determine  the  length  of   p   by  scanning  the 
input  till  it  encounters  a  blank  square.   Chaitin's  de- 
finition essentially  requires  that  such  information  be 
contained  in  p   itself.   Second,  on  a  technical  level 
the  Kolmogorov  definition  does  not  have  a  subadditivity 
property  that  one  tacitly  assumes  when  programming  with 
subroutines.   For  example,  let   s   be  the  string   r  •  t, 
and  let   r*   and   t*   be  the  minimal  programs  for   r 
and   t   respectively.   One  would  like  to  have  a  result 
of  the  type: 

K(s)  <_  K(r)  +  K(t)  +  c  =  |r*l  +  1 1*  |  +  c 

(where   c   is  independent  of   r,s   and   t.)   Under  the 
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THEOREM  1.3.1.   a)   There  exists  a  universal  optimal 
computer   U   in  the  sense  that  BcV'slH  (s)  <_  H  (s)  +  c] 

U  £\ 

where   c   depends  only  on  A  and   U. 

b)   For  any  two  universal  computers 

U- ,    U    :      5c>/s[|H      (s)    -   H      (s)  I    <   c]      where      c      depends 
1        z  ^1  '-'2 

only  on     U        and     U    . 

Hence  we  choose  a  particular  universal  computer   U   and 
define:   H(s)  =  ^  fs) . 


Def.  1.3.3.   s*  =  min  p(U(p)  =  s) . 

We  now  define  the  relative  complexity  of   s   given   t 
again  in  terms  of  a  Turing  model  of  computation.   We 
recall  that  A(p,n)  =  t  means  that  Turing  machine   A 
with  p   inscribed  on  its  input  tape  and   r   on  its 
work  tape  halts  with   t   inscribed  on  its  work  tape  and 
the  reading  head  on  the  input  tape  is  scanning  the  last 
square  of  the  input  p,   never  having  left  the  original 
input. 

Def.  1.3.4. 

/min  IpI 
H^(s/t)  =  U(p,t*)  =  s 

C~  if  Vt(A(p,t*)  7^  s.^ 

As  before  we  get  an  analogue  to  Theorem  1.3.1  and  there- 
by define   H(s/t)  =  H  (s/t) . 
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THEOREM  1.3.2.   For  any  string   t   if   C(p,t)   is  defined 
then  for  all  strings   r  7^  A   C(p«r,t)   is  undefined. 

COROLLARY:   For  any  string   t,   S  =  {plc(p,t)   is  de- 
fined} is  an  instantaneous  code. 

We  now  define  "probabilities"  associated  with  a 
string   s. 

Def.  1.3.5. 

^  C(p)=s  2"^' 


^        C(p,t*)=s  2'^ 


P(s)     =  Pu(s)^^ 
P(s/t)   =  Pu(s/t)^^ 


THEOREM  1.3.3. 


a)   Vs[s*  7^  A] 


b)   0  <  P^(s)  <  1 


c)   0  <  P  (s/t)  <_   1 


d)  H(s)  7^  « 

e)  H(s/t)  7^  <» 

-H_(s) 

f)  P^(s)  >  2   ^ 

-H_(s/t) 

g)  P,.(s/t)  12^ 

h)   0  <  P(s)  <  1 

i)   0  <  P(s/t)  <  1 
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THEOREM  1.3.4. 

a)   For  any  computer   C; 


1  >  I  P  (s)  =    I  ^ 


C(p) defined  2'^' 


b)   For  any  computer   C   and  string   t 

1 


1  >  5:  P  (s/t)  =    I 


C(p,t*)defined  2'^' 


Def.  1.3.6. 


0)  =  u 


=  I   P(s)  =    I 


"   s        U(p) defined  2'^ 


(o)   is  just  the  probability  that  a  computer   U   halts)  . 
THEOREM  1.3.5.   There  exists  a   c   such  that  for  all   s: 

a)  H(s/s)  <_  c 

b)  H(H(s)/s)  <_  c 

c)  H(s*/s)  <_   c. 

Def.  1.3.7.   <s,  ,n,  >  (k  =  0,1,2,...)   is  a  list  of  satis- 
f iable  requirements  iff 

CO 

1)    I  Vi^ 

2)  For  all   k,   n   >  0. 

3)  There  exists  a  recursive  function   F:   N  -»■  X* 
with  infinite  range  such  that   F(k)  =  <s,  ,n,  >. 
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C   is  said  to  satisfy  a  list  of  satisfiable  require- 
ments iff 

1)  For  each   <s  ,n,  >   there  exists  precisely  one 

program  pj^   such  that   I  Pk  1  ^  "^k  ^  '^^Pk^  ^  ^k* 

2)  For  each   p   such  that  e(p)  =  s   there  exist 
a  k   such  that   |p|  =  n   and   s  =  s  . 

THEOREM  1.3.6.   For  any  list  of  satisfiable  requirements 
there  exists  a  computer  C  v/hich  satisfies  them. 

By  H(n)   we  mean   H(t)   where   t   is  the  string 
associated  with   n. 


THEOREM  1.3.7.   a)    T    j;  .  .   converges 

n=l  2"^"^ 


For  any  recursive  function  F:   N  ->■  N; 


°°   1 
b)   If   I        f{^   diverges  (e.g.  F(n)  =  [n]) 

n=l  2 
then  H(n)  >  F(n)   infinitely  often. 


°°   1 
c)   If    1        p ,^\       converges  (e.g. 

n=l  2 
F(n)  =  (1  +  e)In|)   there  exist  a 
c   such  that   H(n)  <  F (n)  +  c. 

THEOREM  1.3.8.   a)   There  exist  a   c   such  that  for  all 
n   if   |s|  =  n   then   max(H(s))  =  n  +  H(n)  +  c. 
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b)   There  exists  a   c   such  that  for  all   n   and  k 

there  are  less  than   2       strings  s   of  length   n 
such  that  H(s)  <_  n  +  H(n)  -  k. 

C)   There  exists  a   c   such  that  for  all   s   and   t: 

t: 

H(S)  <_  H{s/t)  +  H(t)  +  c. 

d)   There  exists  a   c   such  that  for  all   n   if 

|s|  =  n   then: 

H(s)  =  H(s/n)  +  H(n)  +  c. 

1.4.   Elementary  Theorems  Relating  the  Kolmogorov  and 
Chaitin  Formulations. 

THEOREM  1.4.1.   if   K(s)  =  n   then  there  exist  a   c   such 
that  for  all   s: 

H  ( s )  <_  n  +  2  I  n  I  +  c . 

Proof:   If   K(s)  =  n   then  there  exists  a  program  p 
such  that   U(p)  =  s  with   |p|  =  n. 

We  now  construct  a  computer  C   such  that: 

H  (s)  <_  n  +  2]  n|  +2. 

C   computes  as  follows  on  any  input  of  the  form: 


PI  I  •  p  =  |n|  •  p: 

1.  C   first  determines  \p\      moving  its  reading 
head  to  the  last  square  of   | | p{ | . 

2.  C   then  copies  the  next   [p]   symbols  onto  its 
work  tape . 

3.  C   then  simulates   U(p)   without  ever  moving  the 

reading  head  on  the  input  tape  from  where  it 
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was  at  the  end  of  step  2. 
It  is  therefore  clear  that  if   K(s)  =  n   then 


H  (s)  £  n  +  |n I  =  n  +  2|ni  +2.   Hence  since 

H(s)  <_   H  (s)  +  c   we  conclude  that   K(s)  =  n   implies 

H(s)  _<  n  +  2  |nl  +  (c  +  2)  . 

Q.E.D. 

COROLLARY:   If   K(s)  =  n   then   H(s)  <_  n  +  2 1  [n  |  |  +  |  n  |  +  c 
<_  n  +  (1  +  e)  |n|  +  c. 

Proof:   By  an  argument  similar  to  the  one  used  in  the 
above  theorem  we  can  construct  a  computer  C   for  which 

C(| ln| I • ln| -p)  =  U(p)   where   |p|  =  n. 

COROLLARY:   If   K(s)  =  n   then   H(s)  <_  n  +  H(n)  +  c. 

Proof:   If  H(n)  =  k   then  there  exists  a  program  q 
such  that   lq|  =  k   and   U{q)  =  n'   where   n'   is  the 
string  associated  with  the  integer   n.   By  an  argument 
similar  to  the  one  used  in  the  above  theorem  we  can  con- 
struct a  computer   C   such  that: 

C(q  •  p)  =  U{p)  . 

THEOREM  1.4.2.   Let   s*   be  a  minimal  program  for   s 
(either  in  the  sense  of  Kolmogorov  or  Chaitin)  then  there 
exists  a   c   such  that  for  all   s   and   t: 

K(t/s*)  <_   K(t/s)  +  c. 

Proof:   There  exists  a  computer  C  which  on  inputs   p 
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and   s*   computes   U(p,U(s*)).   Hence 


K^(t/s*)  =  K(t/s) 


from  which  we  conclude 

K(t/s*)  <_   K(t/s)  •  +  c. 

THEOREM  1.4.3.   If   K(s/t*)  =  n   then  there  exists  a   c 
such  that  for  all   s   and   t: 

a)  H(s/t)  <  n  +  2  |n|  +  c 

b)  H(s/t)    _<n  +   2||n||    +    jn]    +c<_n+    (1+    e)|n|    +c 

c)  H(s/t)    £  n   +   H(n)    +   c. 

Proof:   The  proof  of  the  above  theorem  follows  the  same 
lines  as  the  arguments  used  to  prove  Theorem  1.4.1  and 
its  corollaries. 

We  now  wish  to  define   H(s/t)   as  a  cross  between 
the  Kolmogorov  and  Chaitin  formulations.   That  is,  we 
will  require  self-deliniation  but  assume  that   U   is 
given   t   and  not   t* .   Recalling  the  discussion  prior 
to  Definition  1.3.4  we  have: 


Def.  1.4.1. 


H^(s/t)  =  iA(p,t)  =  s 


mxn  I p I 

=  s 

if  Vp(A(p,t)  7^  s) 


Ais  before  we  get  an  analogue  to  Theorem  1.3.1  and 
thereby  define   H(s/t)  =  H  (s/t) . 
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THEOREM  1.4.4.   There  exists  a  constant  c   such  that  for 
all   s   and   t: 

H{s/t)  >_  H(s/t)  -  c. 

Proof:   Consider  a  computer  C   defined  by 

C(p,t*)  =  U(p,U(t*))  . 


Then 


H^(s/t)  <  H(s/t)  +  c. 


cind  since 

H(s/t)  <  H^(s/t)  +  c 

we  conclude 

H(s/t)  <  H{s/t)  +  c. 


Q.E.D. 


THEOREM  1.4.5.   There  exists  a  c   independent  of  s 
and   t   such  that: 

H(s/t)  <_  H(s/t)  +  H(t)  +  c. 

Proof:   The  theorem  follows  immediately  from 
H(s/t)  <_  H{s)  +  c   and  Theorem  1.3.8(c). 

THEOREM  1.4.6.   If   K(s/t)  =  n   then  there  exists  a   c 
such  that  for  all   s   and   t: 

a)  H(s/t)  <_  n  +  2  |n|  +  c 

b)  H(s/t)    <_n+2||n||    +    |nl+c<_n+(l+      e)lnl+c 

c)  H(s/t)     <_  n   +   H(n)    +   c. 
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Proof:   Theorem  1.4.6  is  an  exact  analogue  of  Theorem 
1.4.3. 

1.5.   Recursive  and  Recursively  Enumerable  Sequences. 

Of  special  interest  is  the  infinite  sequence   x 
defined  by  x.  =  1   if  and  only  if   i  e  R  where   R  is 
usually  a  recursive  or  recursively  enumerable  set.   x 
is  then  called  the  characteristic  sequence  associated 
with   R.   It  is  obvious  that: 

THEOREM  l.S.l.-'-'^   If   x   is  a  characteristic  sequence  of 
any  recursive  set   R,   then   K(x  /n)  £  c. 

If  x   is  the  characteristic  sequence  associated 
with  a  recursively  enumerable  set   R  then: 

THEOREM  1.5.2.15  ^)       3cVn[K{x^/n)    £  | n |  +  c] 

b)   3cV^n[H(x"/n)  <_  max  H(i)  (i  <_  n)  . 

Proof:   Since   R   is  recursively  enumerable  there  exists 
a  recursive  function   f  which  enumerates   R.   Let   i   be 
the  last  member  of   R  less  than  n   enumerated  by   f. 
In  order  to  determine   x   we  need  only  encode  the  func- 
tion  f   and  the  integer  i,  the  former  requiring  some 
constant  number  of  bits. 

Q.E.D. 

The  following  shows  that  the  bounds  in  the  previous 
theorem  were  tight. 

15 
THEOREM  1.5.3.     There  exists  a  recursively  enumerable 
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R  such  that  if  x   is  its  characteristic  sequence 
then: 


3cVn[K(x^/n)  >  |nl  -  c] . 
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Chapter  2 

INFINITE  RANDOM  SEQUENCES 

2.1.   Basic  Definitions  and  Results  of  the  Kolmogorov 
Formulation. 

We  now  consider  the  problem  of  defining  an  infinite 
random  sequence.   One  might  first  attempt  to  define   x 
to  be  random  if  and  only  if   3c\7h[K(x")  >  n  -  c]  .   This 
however  has  been  shown  by  Martin-Lof   to  be  incorrect 
since  he  has  proven  that  no  sequence   x   satisfies  so 
strong  a  requirement.   Before  noting  Martin-Lof 's  result, 
we  quote  a  weaker  result  in  an  exercise  in  Feller's  text- 
book which  implies  that  the  set  of   x   satisfying  the 
above  requirement  is  a  set  of  measure   0. 

THEOREM:   Let  N^      be  the  number  of  successive  zeros 

n 

ending  in  position   n  of  the  sequence   x.   Then  with 
probability   1: 


N^ 


TIST 2 =  1. 

n->oo  Log2(n) 


By  the  above  theorem  for  almost  all  strings   x   there 
exist  infinitely  many   n   for  which 


^  n-tLog-Cn)]  [Log2  (n)  ] 

x   ~  X  0 


(where   [r]   denotes  the  integer  part  of   r) .   For  any 

such  n   it  is  clear  that   K(x  )  ~  n  -  Log„ (n) .   In  fact 
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the  following  theorem  of  Martin-Lof  holds: 
THEOREM;^   Let   F(n)  be  any  function  for  which 

00 

y   L___  =  00   (e.g.   F(n)  =  Log.  (n)  )  .   Then: 
iil  2^^  2 


\/x3n[K(x^)  <  n  -  F(n)] 


In  light  of  Martin-Lof 's  result  the  following  de- 
finitions of  an  infinite  random  sequence  were  proposed. 

Def.  2.1.1.   X   is  random (1)  iff  5c^n[K(x  )  >  n  -  c] . 

oo 

Def.  2.1.2.   X   is  random  (2)  iff  Jc^[K(x   /n)  >  n  -  c]  . 

Daley^  has  shown  that  x   is  random (1)  if  and  only  if   x 
is  random(2).   We  will,  therefore,  refer  to  a  sequence 
as  K-random  if  and  only  if  it  is  either  random (1)  or 
random ( 2 ) . 

Martin-Lof   and  others  have  shown  that 
C  =  {xjx  is  K-random}   is  a  set  of  measure  1. 

2.2.   Chaitin's  Proposed  Definitions. 

In  terms  of  his  formulation  of  the  notion  of  com- 
plexity, Chaitin  gives  the  following  definition  of  an  in- 
finite random  sequence. 

Def.  2.2.1.^   X   is  C-random  iff   3cVn[H{x'^)  >  n  -  c]  . 

This  definition  of  Chaitin's  relates  to  another  defini- 
tion of  randomness  Chaitin  proposed  for  the  Kolmogorov 

type  complexity. 
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Def.    2.2.2.^   X      is      C -random      iff      Jc^/hEKCx  /n)    > 
n  -    3 |n|    -   c] . 

THEOREM   2.2.1.       If      X      is   C-random   then      x      is   C -random. 

Proof:      If      X      is      C-random  then: 

3cVn[H(x^)    >   n  -   c]  . 

However   3c^n[H(x")  =  H(x'^/n)  +  H(n)  +  c]  (Theorem  1.3. 8.  a) 
and  therefore   x   is  C-random  implies 

3cVn[H{xVn)  >  n  -  H(n)  -  c]. 

However  since  there  exist  a  c   such  that  for  all  n: 

H(x"/n)  >  H(x"/n)  -  c   (Th.  1.4.4)  and 

H(n)  <  (1  +  e) |n|  +  c       (Th.  1.3.7.C) 

we  have  that  if  x   is  C-random: 

(*)       3cV'n[H(x'^/n)  >  n  -  (1  +  e)  |n|  -  c]  . 

By  Theorem  1.4.6(b)  we  have: 

?c\/n[H(x/n)  <  K(x'^/n)  +  (1  +  e)|K(x^/n)|  +  c] 

which  together  with  the  fact  that   3cVn[K(x  /n)  <  n  +  c] 
implies  that: 

3c^n[n{x^)    <  K(x"/n)  +  (1  +  e)|n|  +  c]  . 

Transposing  we  get: 
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ac\7'n[K(x'^/n)  >  H(x  /n)  -  (1  +  e)|nl  -  c] 

which  combined  with  (*)  above  yields  that  if   x   is 
C-random 


3cVn[K(x'^/n)  >  n  -  2(1  +  e)lnl  -  c  >  n  -  3ln|  -  c] 


which  by  definition  implies  that   x   is  C -random. 

Q.E.D. 

2.3.   A  Definition  Based  on  Chaitin's  Formulation. 

Because  of  the  complexity  oscillations  which  the 
theorems  of  Feller  and  Martin-Lof  tell  us  to  expect,  it 
seems  natural  to  attempt  a  definition  of  randomness  of 
the  form:   there  exist  infinitely  many  n   for  which 
H(x  )   is  maximal  similar  to  the  definitions  of  this 
type  given  for   K(x  )   and   K(x  /n) . 

00  _ 

Def.    2.3.1.      X      is   B-random   iff     3c5n[H(x   )    >   n   +  H(n)    -   c] , 

We  now  consider  the  relationship  between  B-random 
and  K-random  sequences. 

THEOREM  2.3.1.   x   is  B-random  implies  that   x   is  K-random, 

Proof:   We  will  assume  that   x   is  B-random  but  K-non  ran- 
dom. 

Negating  the  definition  of  K-randomness  we  get: 

x   is  K-non  random  iff   lim(n  -  K(x  /n)  )  =  <». 

n^oo 

However,  since   KCx'^/n)  >_  K(x"/n*)  -  c  where   n* 

is  an  optimal  program  for   n   in  the  sense  of  Chaitin 
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(Theorem   1.4.2)    we   have : 


X      is   K-non-random   implies:      lim(n   -   K(x  /n*))    =  ". 

n->oo 


On  the  other  hand  since   x   is  B-random  for  infinitely 
many  n   there  exists  a   c   such  that: 

HCx'^)  =  H{x^/n)  +  H(n)  +  c  >  n  +  H(n)  -  c 

and  therefore: 

(*)  lim(n  -  H(x"/n))  <  « 

n->^ 

Now  let  p   be  the  minimal  program  for  x   given 
n*   in  the  sense  of  Kolmogorov.   Then   [pi  =  K(x  /n*) 

and  n-  |p  I  =n-K(x  /n*) . 
'  n 

We  claim  there  exists  a  computer  C   and  a  constant 
c   such  that  for  all   n: 

H^(xVn)  <  K(xVn*)  +  2ln  -  K(xVn*)l  +  c. 

The  construction  of  C  is  similar  to  the  construc- 
tion of  the  computer  used  in  Theorem  1.4.1.   Note  that 
since  .n*   and  hence  n  is  available  to  C   it  can  deter- 
mine the  length  of   p   by  being  given   n  -  | P„ | •   There- 
fore there  exist  constants   c-.  ,  C2   such  that  for  all   n: 

H(xVn)  <  H^(x'^/n)  +  c^  <  K(x"/n*)  +  2|n  -  K(x'^/n*)  \    +   c^. 

But  since  by  assumption 

lim(n  -  K(x^/n*))  =  « 
n->-oo 

we  also  have: 
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lim(n  -  [K{x^/n*)    +  2|n  -  K(xVn*)|]) 


n->-oo 


=  lim([n  -  K(x"/n*)]  -  2|n  -  K(x"/n*)|) 


But  since   H(x"/n)  <_  K(x  /n*)  +  2|n  -  K(x"/n*)|  +  c   we 
conclude: 


lim(n  -  U{x^/n)  )  =  «- 


which  contradicts  (*) . 

Q.E.D. 

2.4.   Recursive  and  Recursively  Enumerable  Sub-sequences. 

Let   f (n)   be  a  recursive  function  with  infinite 
range.   Let  x   be  an  arbitrary  binary  sequence.   Then 
by  x^  we  mean  the  sub-sequence  obtained  from  x  by 
deleting  from  x  every  bit  x.   for  which   i  ^  Range [f(n)]. 

THEOREM  2.4.1.   If   x   is  K-random  and  the  range  of   f (n) 
is  recursive  then  x    is  K-random. 

Proof:.   Since  the  range  of   f(n)   is  recursive  we  assume 

without  loss  of  generality  that   f(n)   enumerates  its 

range  in  increasing  order.   Let  j  (n)   equal  the  number 

of  elements  of  the  range  of   f(n)   less  than   n.   By 

X    we  will  mean  the  sequence  obtained  from  x  by  de- 
°f 

leting  from  x   all  bits   x.   for  which   i  e  Range[f(n)]. 

We  will  now  assume   x    to  be  K-non-random  and   x   to  be 
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K-random  and  derive  a  contradiction. 

Let  A   be  the  binary  encoding  of  a  program  which 

enximerates   the   range   of      f(n). 

Consider  the  following  program  p    for   x  : 


^    f 


Computer  C  operates  as  follows  on  input   p   given  n: 

1.  Evaluate   f(l),  f(2)  ...   iteratively  until 
f(k)  >  n. 

2.  Record   f(l),  f(2),  f(k-l)   and  set   j (n)  =  k-1. 

3.  Reserve   n   cells  on  which  to  record  the  output 

n 
X  . 

4.  Place  each  of  the   n  -  j (n)   bits  following   a7 
successively  into  each  cell   i   for  which 

i  ^   Range [f(n) ] . 

5.  Simulate   U  on  the  remaining  bits  of  the  input 
placing  the  output  successively  into  the  empty 
output  cells. 

Now  if  X£   is  non-random  then   lim( j  -  K(x^))  =  °o. 

Therefore 

lim(n   -    Ipj^D    =   lim(n   -    [c   +    (n   -   j  (n)  )    +   K(x^^"^]) 


=   lim(j(n)    -    [K(xj^^^    +   c]) 


n->"3 


which  implies   x   is  non-random. 

Q.E.D. 
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[Note  that  in  the  above  proof  we  used  both  versions  of 
the  definitions  of  K-randomness  deriving  a  contradiction 
from  the  absolute  non-randomness  of   x^   and  the  condi- 
tional randomness  of   x.]. 

The  above  theorem  illustrates  one  example  where  a 
constant  length  prefix  allowed  us  to  delineate  between 
two  separate  parts  of  a  program.   Of  course,  we  made 
crucial  use  of  the  fact  that  the  range  of   f(n)   was  re- 
cursive.  If  the  range  of   f(n)   is  not  recursive  we  can 
by  an  argument  similar  to  the  above  conclude: 


3c3n[K(xJ)  >  n  -  (1  +  e)|K(x^)|] 


since   (1  +  e) |k(x£) |   bits  might  be  needed  to  delineate 
between  the  part  of  the  program  used  to  compute   x^   from 

the  part  used  to  compute  x^,       where   j  (i)  =  n. 

f 
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Chapter  3 
AN  ANALYSIS  OF  K(s*/s) 

3.1.   The  Set  of  Optimal  Programs. 

We  begin  this  chapter  by  considering  the  set  of 
optimal  programs.   We  will  show  that  set  of  optimal  pro- 
grams is  immune.   Though  we  will  assume  throughout  this 
section  that  we  are  dealing  with  the  absolute  complexity 
formulation  of  Kolmogorov,  the  basic  theorems  remain 
valid  for  all  of  the  complexity  formulations  discussed 
in  Chapter  one. 

Def.  3.1.1.   OP  =  {p|3s[(U(p)  =  s)  &  (Vq  <  p){U(q)  7^  s]  } 

THEOREM  3.1.1.   ^ct^[p  e  OP  ->  |K(p)  -  |p||  <  c]   (i.e. 
p  e  OP   implies   p   is  random) . 

Proof:   Consider  a  computer  C   defined  by: 

C(p)  =  U(U(p))  . 

Let   p  =  t*   and   s  =  p* .   s  =  p*   implies 
3cVp[|s|  <_  IpI  +  c]  .   But  since   C(s)  =  t,   K  (t)  <_  Is| 
and  therefore   |p|  =  K(t)  <_  |s|  +  c^  ^   [where  c^  y 

depends  only  on  C   and   U] .   Hence   |Ip|  -  |s||  = 
I  IpI  -  K(p)  I  <_  max{c,  c^   y}. 

Q.E.D. 

THEOREM  3.1.2.   a)   OP   is  not  recursively  enumerable. 

b)   If   S   is  any  infinite  r.e.  set 
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then   S  n  OP   is  infinite  (i.e.   OP   is  immune) . 

Proof:   OP   is  infinite  since   s*  =  t*   implies   s  =  t 
and  therefore  b)  implies  a) .   Hence  we  need  only  prove  b) . 

Let  F   be  an  algorithm  that  enumerates   S  without 
repitition,  i.e.   S  =  {F(0),  F(l),  ,  F(n),  ...}. 

Define   G(n)   by: 

G(0)  =  F(0) 

G(n+1)    =  F[yj[|F(j)|    >  max{n   +   1,     |G(n)[}]]. 

Claim  1.   G  enumerates  a  set   S'   without  repitition. 

Claim  2.   S' C  S. 

Claim  3.   S'   is  infinite. 

Proof:   Since  the  range  of   F(j)   is  infinite  there  will 
always  be  a  value  of   j   for  which   |F(j) |   is  greater 
than  max{n  +l,|G(n)|}. 

Claim  4.   If   s  =  G(i)   for  any   i   then   |s|  =  |g(i)|  >  i, 
Claim  5.   K^(s)  =  Kg(G(i))  <  |i|. 

Claim  6.   K{s)  <_   Kq(s)  +  c^^  ^  _<  |  i  |  +  c. 

Hence  for  any   seS'   K(s)<^||s||+c   which  implies 
that  all  but  a  finite  number  of  the  elements  of   S'   are 
also  in  OP. 

Q.E.D. 

3.2.   K(s*/s) . 


A  major  difference  between  the  Kolmogorov  and  Chaitin 

formulations  arises  when  we  consider   K(s*/s)  and 
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H(s*/s) . 

By  Theorem  1.3.4.C   H(s*/s)  <_  c  where  c   is  in- 
dependent of   s. 

However   K(x*/x)   behaves  quite  differently.   First 
we  state  a  somewhat  obvious  result  for   K(s**/s). 

THEOREM  3.2.1.   If   ||s|  -  |s**|I  =  c'   then 
3cVs[K(s**/s)  <  Ic'  I  +  c]  . 

Proof:   Define  a  computer  C   as  follows:   Given   s   and 
Oc'(lc')   as  input,   C   iteratively  simulates   k   steps 
[k  =  1,2, . . . ,n, . . . ]   of  the  computation  of   U  on  each  of 
the  2'^'"^' (2'^l"^^')   sequences  of  length   |s|  -  c'(|s|  +  c') 
until  it  finds  an  input  on  which   U  outputs   s   in  the 
least  nxamber  of  steps.   If  it  finds  more  than  one  input 
which  outputs   s   in  the  minimum  number  of  steps  it  chooses 
the  first  with  respect  to  the  ordering  of   X*.   Hence 


K^(s**/s)  <  1  +  |c' 


which  implies 

K(s**/s)  <  |c' I  +  c. 

Q.E.D. 

COROLLARY:   K(s**/s)  <_min{ls**|,  c'}  +  c. 

Note  that  for   s*   even  a  result  as  weak  as  the  above 

may  not  be  true,  since  one  could  not  guarantee  that 

having  found  a  program  p   such  that   |pl  =  K(s)   and 

U(p)  =  s   that  there  does  not  exist  another  program  p' 

such  that   Ip'I  =  |p|,  p'  <  p  and   U(p')  =  s. 
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P-n  unproved  conjecture   is  that  there  exists  a   c 
such  that  infinitely  often   K(s*/s)  >  |s*|  -  c. 

We  now  prove  a  weaker  result  in  the  direction  of  the 
conjecture. 

THEOREM    3.2.1.       %  5c\/s  [K  (s*/s)     <_  c]  . 

c+1 
Proof:   Assume  such  a  c   existed.   Let   r  =  2     and 

let   c  ,  c  ,  ...,  c   be  a  listing  of  all  strings  of 

length  less  than  or  equal  to   c.   Then  for  every   s 
there  is  a   c-   such  that: 


U(c^,s)  =  s*. 


We  now  non-constructively  construct  an  algorithm 
which  will  output  only  optimal  programs,  hence  contra- 
dicting the  immunity  of  the  set  OP. 

Define   # (s)   as  the  number  of   c.'s,  0  <  i  <  r 

1     —   — 

for  which  both  U(c.,s)   and   U(U(c.,s)   halt.   Let 
k  =  lim  sup(#(s).   Then:   1)   there  exist   s'   such 
that  for  all   s  >  s',  #(s)  _<  k,   2)  there  exist  in- 
finitely many   s   such  that   # (s)  =  k. 

We  now  provide  an  algorithm  A  which 

1)  is  undefined  if  either  s  £  s'   or 
#(s)  <  k 

2)  outputs   s*   if   #(s)  =  k  and   s  >  s'. 
A  computes  as  follows: 

1)  If   s  £  s'   A(s)   is  undefined 

2)  If   s  >  s'   dovetail  the  computation  of 
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U(U(c.  ,s)   for   0  1  i  1  r. 

3)   If  and  when   U(U(c.,s))   has  halted  for  k 

different  values  of   c.   list  each  U(c.,s) 

1  1 

for  which   U(U(c.,s)  =  s.   Determine   s*   by 

finding  the  first  element  in  lexocagraphic 
order  on  the  list.   Print  s*. 
A  dovetailing  of  the  computation  of  A(s)   provides 
an  algorithm  which  outputs  an  infinite  list  of  optimal 
progrcims . 

Q.E.D. 
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Chapter  4 
0)   -  THE  PROBABILITY  THAT  A  UNIVERSAL  COMPUTER 

IN  THE  SENSE  OF  CHAITIN  HALTS 


In  Chapter  I  we  defined   u  =  cj„  =  J[  P(s)  = 

s 

U{p) defined  2'  '.   We  now  show  that  in  one  sense  it 

can  be  said  that  a  given  universal  computer  is  preferable 

to  another  in  the  sense  that  H  (s)  <  H   (s) .   This  is 

accomplished  by  showing  how  one  can,  given  any  universal 
computer   U,   find  another  which  while  being  defined  on 
the  same  number  of  inputs  chooses  those  inputs  to  be 
shorter  and  hence  has  a  higher  probability  of  halting. 

THEOREM:   Let  U  be  a  particular  universal  computer  for 
which  1  -  ""jTTT  <  Wy  <  1  -  — jT.   Then  there  exist  another 

universal  computer   U'   such  that 

1.  Hy, (s)  <  Hy(s) 

2.  For  all   p   sufficiently  large   U(p)  =  s 
implies  3P'[(|p'l  <  IpI)  &  U' (p')  =  s] 
and  hence, 

3.  For  all   s   sufficiently  large   H   (s)  <  H  (s) . 

U'        U 

4.  0)  .  >  1  -  i^. 

U-        2^ 

Proof:   We  first  prove  (1)  and  (2) .   Let   A  be  an 
algorithm  which  enumerates   <s.,  n.>   where   s  ,  s  ,... 

XX  X     ^ 
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is  a  list  of  all  outputs  of   U   and  n.   is  the  length 
of  the  progrcim  which  gives  output   s- . 


03 


=  I  ^ 

Lj  t 


"   iil,"! 


Since  u)„  >  1  -  .  _..  3i   such  that; 


^0 


(*) 


I  —  >  1  -  Vt- 

1=1  .^i       2^-^ 


Clearly  if   U(p)   is  defined  and   <U(p),  |p|>   is  output 

by  A  after   <x^  ,  n^  >   then   |p|  >  k.   Otherwise 

0     0 


"'Xt^^iik^^^^'^-N^F^^-? 


Let  m  =  max{nj^|i  £  in^*   Define  an  algorithm  A'   which 
enumerates 

As.  ,  n.  >     i  <  i^ 
j   1    1        -  0 

<s:,  n:>  =/  ^        .,  . 

1    1    (  <s^,  n^  -  1>  1  >  In 


"   1 

Claim:    T   -—7-  <  1. 
. '-,   n'. 


i=l  2  i 


Proof; 
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i=l  .""i 


00  ^0  «  0 

0  ,  0  T 

=   I  i—  +  2  fu   -  I      i— 1 
>,   n.     ^""u    >,   n.-' 


00 

+  2   I 


i=i„+l  ^^i 
0    2 


i=l  2  i 


i=l  2  i 


0  ,  ,       0-  , 

=  20)   -    y   i—  <  2(1  -  i^-]  -   y   i— 

u    >,   n.     ^     ^k''    .^,   n. 


i=l  2  i 


i=l  2  i 


=  2 


-  ^-  -   J  ^ -  1 

^k-1    .^,   n.    ^ 


i=l  2  i 


<  1. 


1  - 


.k-1 


0  1 

-   y  ^^ — 

>,   n. 

1=1  2  1 


So  by  Theorem  1.3.6  there  is  a  universal  computer 
satisfying  the  set  of  requirements  generated  by  A' . 

This  proves  (1)  and  (2)  with  "all   p   sufficiently 
large"  in  the  statement  of  (2)  replaced  by  "all 

P  3  IpI  >  m" 

(3)  follows  immediately  from  (2) . 

(4)  To  prove  (4)  consider  2  possibilities: 


a)   If   ojy,  >  1  - 


then  we  are  through. 


b)   Otherwise  we  iterate  the  process.   Clearly 
U'   satisfies  the  hypotheses  of  the  theorem. 
Let  A'   replace  A  as  the  algorithm  to 
enumerate  all  pairs   <s!,  nl>   outputted 

IX 


by  A' 


Note  that 


u  1 

2 
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i=l  2  1 


0  1 

^^u~  I  .  -IT 
^     1=1  2  1 


Hence  the  process  iterated  gives: 


Wy.  . 


1 

^        n 
1=1  2  i 


+  4 


T   1 


i=l  2  1 


If  w  ,  ,  >  1  -  —77     we  are  done  otherwise 
U         2 


0) 


U' 


*o  1 

.  ^.   n. 
1=1  2  1 


+  8 


7  1 


i=l  2  1 


So  that  eventually 


0) 


0  1 


+  2 


n 


U"'"7r^   i=l  2  1 


(A). 


u 


°  1 

.  ^-   n. 
1=1  2  1 


will  exceed  1  -  —rr, 

2^ 


Q.E.D. 


COROLLARY:   lim  sup  w    =  1. 

i 


Proof:   By  (4)  of  the  previous  theorem  we  can  come  as 
close  to  1  as  we  like. 

The  above  result  implies  that  there  are  no  optimal 
universal  computers  in  the  strict  sense. 
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Chapter  5 
SUBRECURSIVE  FORMULATIONS  OF 
DESCRIPTIVE  COMPLEXITY 

5.1.   Introduction  and  Fundamental  Definitions 

We  now  wish  to  consider  the  notion  of  descriptive 
complexity  in  a   subrecursive  formalism.   Though  our 
discussion  is  carried  out  entirely  in  the  context  of  the 
Grzegorczyk  hierarchy,   our  result  will  be  valid  in  any 
hierarchy  which  shares  certain  fundamental  properties 
with  the  Gyzegorczyk  hierarchy.   We  first  give  a  list- 
ing of  those  properties. 

To  briefly  outline  the  structure  of  the  Gyzegorczyk 
hierarchy  consider  the  sequence  of  functions: 

fo(x,y)  =  y  +  1 

f^(x,y)  =  X  +  y 

f2(x,y)  =  (x+l)(y+l) 

fn+i(o,y)  =  fn(y+i'  y+1) 


For   n  =  1,2,  . . . ,  c"   is  smallest  class  of  functions 
such  that : 

(1)  £,^     contains   S(x)  =  x+l,  I^(x,y)  =  x, 
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(2)   5    is  closed  under  composition  and  limited 
recursion.   [a  class  <x  is  closed  under  the 
operation  of  limited  recursion  if  for  any  func- 
tions  g,h,i  which  are  in  class  CZ.   the  func- 
tion  f   defined  by: 

f(x,o)  =  g(x) 

f  (x,y+l)  =  h(x,y,f  (x,y)), 

and  satisfying: 
f  (x,y)  <_  i(x,y) 

is  also  in  class  Cl.'\ 

We  note  the  following  results  concerning  the  hierarchy: 
1)   5    is  closed  under  bounded  minimalization. 

2)  e<^  e^^  ^ 

3  1 

3)  f;-^   is  the  class  of  elementary  functions 

4)  For   n  >  2,^     contains  a  universal  function 
for  the  one-place  functions  of  class   ^  . 

5)  f   , (x,x)   majorizes  all  one-place  functions 

of  level   C    in  the  sense  that  given   f  e  ^'^ 
3cVx[f^^j^(x+c,x+c)  >  f(x)]   for  all   x.-"- 

6)  There  exists  an  infinite  chain  of  classes  of 

algorithms  written  in  a  suitable  formal  language 

12    3 
which  we  denote  by   E  ,  E  ,  E   . ..   with  the 

following  properties: 
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a)  If   f   is  a  function  in   ^    then  there  is 

an  algorithm  A  e  E   which  computes   f. 

b)  For  any  algorithm  A  e  E   the  function 

computed  by  A   is  in  Grzegorczyk  class  E,    . 

c)  Given  any  algorithm  A  and  integer   i   it 


'>  u 


is  decidable  whether  A  e  E  . 


E,^  =   PR,  the  class  of  primitive  recursive 


00 

function  and  hence   ^  E   contains  a  class 

n=l 

of  algorithms  for  the  primitive  recursive  func- 

2 

tions. 

8)   For  any  recursive  function  g(x),   if   g(x)  <_  f(x) 
and  f(x)  e  C   then  g(x)  e  C  • 

It  is  important  to  note  that  many  algorithmic  formalisms 

(e.g.  Turing  machines  or  defining  equations)  do  not  satis- 

2 

fy   6)  c)  above.   However  Minsky  and  others   exhibit  al- 
gorithmic formalisms  based  on  simplified  programming  langu- 
ages which  contain: 

1)-  A  class  of  elementary  instructions  such  as  succes- 
sor and  identity. 

2)  Conditional  forward  transfers. 

3)  Looping  instructions  in  which  the  parameters 

governing  the  number  of  times  the  statements  in 

the  scope  of  the  loop  are  to  be  executed  cannot 

be  changed  by  any  statement  within  the  scope  of 

the  loop. 
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4)   Conditional  backward  transfer. 

Programming  languages  allowing  for  all  four  types 
of  instructions  provide  formalisms  for  the  full  class  of 
partial  recursive  functions  while  programming  languages 
without  conditional  backward  transfer  and  restrictions 
on  the  depth  of  nesting  of  loop  instructions  provide  a 

formalism  for  the  Grzegorczyk  hierarchy  which  satisfies 

2 

the  decidability  requirement  of   6)  c)  above. 

Let   b(n):   N  -»■  X*   be  the  function  which  maps   n 
into  the   nth  element  of   X*.   Let   b~  (n)  :   X*  -»-  N  be 
its  inverse  with   B   and   B    algorithms  for   b (n)   and 

b   (n)   respectively. 

Let  hi,    hi,    ....  A'   be  a  list  of  algorithms  for 
0    1        n 

all  partial  recursive  functions:  N  ->-  N.  If  A.  =  BA!B 
then  Aq,  a,,  .../  A  ,  ...  is  a  list  of  all  algorithms 
for  partial  recursive  functions:   X*  -*   X*. 

Def.  5.1.1.   A.  £  E^   iff   A!  e  E^ . 


"At     c-    TlD  II 


Note  that   "A^  e  E-^ "   is  decidable  since   "A!  c   E 

is  decidable  by  requirement  6)  c) . 

Relative  to  a  particular  listing  of  all  algorithms 
Aq,  a,,  h~ ,    ...,  A  ,  ...   we  define: 


U(o^lp)  =  A^(p) 


as  the  universal  computer  on  which  our  treatment  is  based, 
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We  now  define  a  universal  optimal  computer  for  each 
level   J  ^  3   of  the  hierarchy.   [All  results  in  this 
chapter  are  claimed  only  for  levels   C  »  i  >^  3 .  ]  . 


if   A.  e  E 

1 

otherwise 


For  functions  of  two  variables  we  can  similarly  define 
a   jth   level  universal  optimal  computer 


In  terms  of  the  universal  optimal  computers  of  one 
and  two  variables  we  define  the  notion  of  the  descrip- 
tive complexity  of   s   relative  to  level   j   of  the  hier- 
archy.  Note  that  since  the  identity  computer  is  present 
at  every  level  of  the  hierarchy  the  complexity  of  any 
string   s   is  never  infinite. 

Def.  5.1.4. 

a)  K^  (s)  =  min  |p|  [U^  (p)  =  s] 

b)  K^(s/t)  =  min  ip|[U^(p,t)  =  s] 

c)  H^  (s)  =  min  |pl  [U^  (p)  =  s   where   p   is  self  delimit- 

ing as  previously  explained] 

*  * 

d)  s  ^  =  min  p[U-'  (p)  =  s]  .    Note  that   s  ^      will  be  used 

to  refer  to  the  minimal  program  for   s   in  either  the 
Kolmogorov  or  Chaitin  formulation.    This 

should  be  clear  in  context. 
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THEOREM  5.1.2.   x   is  i-random{l)  iff   x   is  i-random(2). 

As  a  result  we  will  define  x   to  be  K. -random  iff   x   is 
i-random(l)  or  x   is  i-random(2) . 

The  statement  and  proof  of  Theorem  2.3.1  carries 
over  directly.   Hence  as  before  we  can  prove: 

THEOREM  5.1.3.   X   is   B . -random r^  x   is  K. -random. 

While  the  converse  implication  was  left  unsettled  in 

Chapter  two,  in  the   next  section  we  will  show  that: 

X   is   K.., -random  implies   x   is  B. -random. 
1+1  ^  1 

*  _ 

5.2.   K^(s  ^/s)   and  Related  Results. 

We  showed  earlier  (Theorem  3.1.2)  that  the  set  of 
optimal  programs  was  immune.   We  now  show  that  in  a  sub- 
recursive  formalism  OP.   and  hence   OP.   is  recursive. 

1  1 

Def.  5.2.1.   OP.  =  {p|3t[(U^(p)  =  t)  &  (Vq  <  p)U^(q)  7^  t]  } 

THEOREM  5.2.1.   OP.   is  recursive. 

Proof:   Given  a  program  p  we  evaluate   U  (p)   and  then 
check  U^(q)  for  every  q  <  p   to  see  whether  or  not 

U^{q)  =  U'^  (p)  .   Note  that   U"""  (p)   is  in  Grzegorczyk  level 

*  _ 

E,  and  therefore  the  predicate:   "p  =  s  -' "   is  a  rela- 

lation  in  Grzegorczyk  level   C 

Q.E.D. 
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COROLLARY:   There  exist   c   such  that  for  all   s 

* 

a)  K^'^^Cs  ^/s)  <  c 

* 

b)  U^'^^is   ^/s)  <  c. 


THEOREM  5.2.2.   x   is   K., .-random  implies  that   x   is 

1+1  ^ 

B. -random. 

1 

Proof:   Clearly  ^cVnEK^"^^  (xVn)  <_  H^"^""- (xVn)  +  c]  .   By 

* 

the  above  corollary   n  ^   can  be  obtained  from  n  by  a 

fixed  length  program  and  therefore: 


gcVn[H^'*'-'-(xVn)  <  H^(xVn)  +  c] 


Hence: 


BcVn[K^      (x^/n)  <  H^(x"/n)  +  c] 


Now  assume  that  x   is   K^^, -random  but  not  Bj^-random. 
If  X   is  B- -non-random  then 


lim(n  -  H^(x^/n))  =  <=° 


but 


3cV'n[K^'^^(x^/n)  <  H^Cx'^/n)  +  c] 


implies  that 


lim(n  -  K^'^^ix^/n))    =  » 


n-»-o 


-47- 


which  contradicts  the  assumption  that  x   is   K   -random 

Q.E.D. 

5.3.   A  Hierarchy  of  Random  Sequences. 

Given  that  x   is   K. -random  (B. -random)  it  is  clear 

1  1 

that  X   is  a  fortiori   K.  .-random  (B.  , -random) .   It  is 
—  1-1  1-1 

unclear  whether  the  converse  is  valid.   We  think  not  and 
in  this  section  outline  a  construction  which  given  a  se- 
quence  X  which  is  K-random  (B-random)  constructs  an- 
other sequence   y  which  is  K.,, -non-random  (B., -non- 
random)  but  which  we  conjecture  is  K. -random  (B. -random). 

If  our  conjecture  is  proved  false  and  it  is  further  proved 

that  X   is  K. -random  (B. -random)  implies  that   x   is  also 
11 

K-random  (B-random)  then  each  of  the  following  results 
would  follow: 

1)  X   is  K. -random  implies  that   x   is   K. ..-random 

1  '^  1+1 

2)  X   is  K. -random  implies  that   x   is  B. -random 

1  1 

3)  X   is  K-random  iff   x   is  B-random. 

Note  that: 

1)  is  weaker  than  the  hypothesis 

2)  follows  from  1)  and  Theorem  5.2.2. 

3)  follows  from  the  following  chain  of  equivalences: 
X   is  K-random 4^  X   is  K . -random ^r^x   is  B. -random 

<;^  X   is  B-random. 

We  feel,  however,  that  the  following  conjecture  is 
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CONJECTURE:   There  exists  a  sequence   y  which  is 
Kj-random  (B--randora)  and  is  not  Kj^^.^^- random  (Bj^^-j^-random) 

We  give  the  following  construction  in  support  of  the 
conjecture : 

Let  X  be  a  random  string  (K-random  or  B-random) . 

Let   F  be  a  function   N  ->  N   such  that   F  e  C""""*^   but 

F  dominates  any  function   g  e  5"'".   Define  a  string   y 
as  follows: 

,  ,        "^o+l    '^o 
Let  n  =  F(o)   and  y     =  x  1. 


"^o 


Compute  F  (x  1)  =  n-i   and  set 


n-+2    n 
y     =  X  °lx     ...  X  1 

^o        ^1 

=  Y    x^^^^  . . .  x^^l  . 

Continuing  iteratively  we  let  n   =  F(y      )   and  set 

nj^+(k+l)     n^-i+k 
y        =  y      X      ....  X  1. 

^-1        ^ 


This  defines  a  sequence   y  which  is  clearly  non-random 
at  level   i+1   since  a  program  for  y    is  given  by  p  ■ 


c 


bits  to  encode  the  function   F  •  x^"-^ 


where   j (n)   is  a  count  of  the  number  of   I's   inserted 

n-j (n)  n 

in  X        to  give   y  .   Hence 
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lim(n   -    Ip|)       =   lim(n   -    (n   -    j  (n)    +   c))   =    lim(j  (n)    -   c)    =   ", 


A  similar  argument  shows  that  y   is  also  B.   -non-random. 
We  are  unable  to  shov;  that   y   is  K. -random  or  B. -random 
and  the  conjecture  remains  unproved. 

5.4.   The  Maximum  Grr:egorczyk  Level  of  All  Programs  of 
Length  Less  Than   n. 

In  this  section  v/e  prove  a  theorem  based  on  Blum's 
fundamental  theorem  on  the  size  of  machines  (algorithms) 
which  clearly  underlines  an  important  distinction  between 
the  Grzegorczyk  hierarchy  and  the  hierarchy  of  classes  of 
algorithms  discussed  in  section  one.   Let  A  ,  A  ,  A  ,  ... 

A  .  .  .   be  an  effective  Godel  ni.imbering  of  all  algorithms 
n 

in  a  Minsky-type  programming  language  formalism  for  the 

set  of  partial  recursive  functions. 

Def.  5.4.1.    A  recursive  function   s   mapping   N   (viewed 
as  the  set  of  indices)  into   N   (viewed  as  the  set  of 
sizes)  is  called  a  size  measure,   s(A^)   denoted  by   |i| 

being  called  the  size  of  A.   iff 

1)  There  exist  at  most  a  finite  number  of  machines 
of  any  given  size  and 

2)  there  exists  an  effective  procedure  for  deciding, 
for  any   j ,   which  algorithms  are  of  size   j . 

[Note  that  in  this  section  only  we  will  follow  Blum's 

convention  and  let   |i|   denote  the  size  of  the   ith 
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algorithm. ] . 

THEOREM  5.4.1.   (Blum's  fundamental  theorem  on  machine 
size)  . 

1.  Let  g   be  any  recursive  function  with  infinite 
range. 

2.  Let   f  be  any  recursive  function.   Then  there 
exist   i,j  e  N,   both  uniform  in   f,g   such 
that: 

a)  A.  =  Ag(j) 

b)  f|i|  <  |g(j)|. 

Consider  all  algorithms  of  size  n  or  less.   By 
the  axioms  a  size  measure  must  satisfy  there  are  at  most 
a  finite  set  of  algorithms  whose  size  is  less  than   n. 
Hence  there  exists  an  effective  procedure  to  determine 
the  maximum  class   E"""   in  which  any  algorithm  belonging 
to  any  E-'   lies.   Similarly  there  are  at  most  a  finite 
number  of  functions  computed  by  an  algorithm  of  size  less 
than  n.   Hence  among  those  functions  computed  there  must 
exist  at  least  one  function   f   such  that  if   f  e  C 
then  for  any  other  function  g   computed  either   g   is  not 
primitive  recursive  or   g  ^  C^.     We  summarize  the  above 
with  the  following  definition: 

Def.  5.4.2. 

fmaxCi)   f   is   1.  primitive  recursive,  2.  com-   ^. 
i        puted  by  a  algorithm  of  size  less  than    \ 
n,   and  3.  contained  in   ^i. 
G(n)  =  ; 

0       if  no  primitive  recursive  function  is  com 

puted  by  a  program  of  size  less  than   n. 
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THEOREM  5.4.2.   G{n)   is  not  recursively  bounded. 

Proof:   Assume   G(n)  <  h, (n)    for  some  recursive  func- 
tion  h  (n)  .   Define   h  (n)  =max[h^(n),  h^  (  |  n  |  )  ]  . 

(1)  G(n)  <  h2(n) . 

For  every  k   define   hj  (k)  =  maxIhjC^j^),  ...f  ^2(^^)1  +  1 

where  I    ,    I    ,    . . . ,    I        are  the  indices  of  all  programs 
12        m 

of  size  k. 

(2)  h^dnl)  >  h^Cn)  >  G(n)  . 

(3)  \:'i,j   if  f.  e  C.   and   ji]  =  k   then 

then  h^Ci)  >  h^  (  [  i  |  )  =  h^(k)  >  G(k)  >  j. 

(4)  Putting  (2)  and  (3)  together  we  get 
h^li]  >  j   where  f^   e    E,.. 

Now  let  g{n)   enumerate  the  canonical  indices  of 

f   where   f    is  the  intial  function  added  at  level   C„ • 
n  n  n 

g(n)   is  strictly  monotone.   Define   2-(n)  =  |g(n)l. 

Jl(n)   is  likewise  a  strictly  monotone  recursive  function. 

Finally   h.  =  il  o  h_. 
4         3 

We  now  apply  Blum's  theorem  to   h   and  g: 

Given  g   a  recursive  function  with  infinite  range  and 
h^   as  above  there  exist   i,j   such  that 

a)  fi(x)  =  fg(j)(x). 

b)  h^(|i|)  <  |g(j)  I  =  il(j)  . 
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Combining  b)  and  (4)  we  have 

(4)   h3|i|  >  j   since  f .  =  f^^.^    e    K- 

b)    iKh^dil))  <  Z{j)  . 

Since  a  is  monotone  we  can  remove  i  from  both  sides 
of  the  inequality  giving: 

b')   h- ( I i I )  <  j   which  is  the  desired  contradiction, 

Q.E.D. 

5.5.   The  Longest  String  of  Complexity  Less  Than   n. 

While  results  in  this  section  are  stated  in  terms 
of   K(s)   and  K^(s),   analogous  results  hold  for   H(s) 

and  H  (s) .   For  the  remainder  of  this  section  we  denote 
the  string  associated  with   n  by  n'. 

Let  A  be  an  algorithm  for  a  function   f:  X*  -*•  X*. 

Def.  5.5.1.   K(A)  =  min  |s| [U(s)   is  a  binary  encoding 
of   A]. 

Chaitin  has  defined   two  functions  which  are  analo- 
gous to  Rado's   "busy  beaver"  functions. 

Def.  5.5.2.   a(n')  =  max  s  [K(s)  <_  n]  . 

Def.  5.5.3.   b(n')  =  max  A(n' ) [K (A)  <  n] 

[Note  that  a(n')  &  b(n')  may  not  be  defined  for  small 
values  of  n'  since  there  is  no  guarantee  that  the  set 
of  s  for  which  K(s)  <_  n  or  the  set  of  A  for  which 
K(A)  <_  n   is  non-empty.]. 
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.  .  7 

Chaitin  has  shown: 


THEOREM   5.5.1.       a)      90^11  [a  (  (n+c)  ' )    >    b(n')     &   b((n+c)')    >    a(n')] 

b)   a(n')   and  hence   b(n')   dominate  all  re- 
cursive functions. 


We  now  analogously  define   a. (n')  .  and  b. (n') . 


Def.  5.3.4.   a.  (n')  =  max  s(K^{s)  <_  n)  . 

Def.  5.3.5.   b. (n')  =  max  A(n') [K^(A)  <  n  &  A  e  E^] 

1  — 

We  now  wish  to  show  that  a^(n')  z    E,  but  not  in 


THEOREM  5.5.2.   a.(n')  ^  ^^. 


1 


Proof:   Assume   a^(n')  e  E,    ,      then   a*(n')  =  l*a^(n') 
is  also  in   C  • 


Let 


A^   be  an  algorithm  for   a*(n').   Consider 


,,i  ,Jfi,   ,  n-  (k+1)  ,   ^      ..  1  j^T 
U  (0  1  1       )   for   n  >  k+1. 


,k  ,  ,n-(k+l)  I  _ 


iQK^^n-^K-M;!  ^  ^^ 


yi(0kiin-(k+l))  ^  ^  (in-(k  +  l)^  ^  a*(l""^'^''^M 


=  1  •  a.  d"  ^^"^^^).   Since  the  string   l"^  ^^"'"■'■^   corres- 

n-k 
ponds  to  the  integer   2    -  2 

U^O^  1  i^-(k+l))  =  1  .^  (in- (k+1)  )  >  ^  („.), 


which  contradicts  the  definition  of   a. (n) . 

~^^~  Q.E.D. 


THEOREM:   a^(n')  e  C^"*"  • 

Proof:   We  show 

1)  b^(n')  e  C^"^-"- 

2)  3cVn[a. (n')  <  b^((n+c) ')] . 

Proof  of  1) : 

Consider  the  following  algorithm  for  b^(n'). 

«    J.        ^n+l 

1.  Compute   2 

2.  Check  each  of  the  2^'^-^  sequences  t  of  length 
less  than  or  equal  to  n  to  see  if  U^(t)  is  a 
binary  encoding  of  a  program  in   C  • 

3.  For  each  t  for  which  U(t)   is  the  binary 
encoding  of  an  algorithm  A  e  C   compute 
A(n'). 

4.  Output  the  maximum  A(n'). 

Clearly  each  of  the  above  steps  can  be  performed 

-i+1 
in  5    . 

Proof  of  2)  : 

Consider  A,-  _.   an  algorithm  for  the  constant 
1  /  n 

function  whose  value  is  always   a.(n'). 

Since   K^{A.   ,)  <  K^(a.(n))  +  c  <  n  +  c   we  can 
i,n'   —     1         ~ 

conclude  that 


-55- 


a    (n')    =  A.       ,  {  (n  +   c)  ')    <_  max  A(n')  [A(n')    e    C^    & 

K^  (A)     <_  n   +   c]    =   b    (  (n   +   c)  • )  . 

Q.E.D, 
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NOTES 
Chapter  1. 

1.  The  fundamental  papers  on  which  most  of  this  section 
is  based  are  Chaitin  [4]  and  [5] ,  Kolmogorov  [13] 
and  [14] ,  Solomonoff  [22]  ,  and  Martin-Lof  [17] .   For 
a  more  complete  introduction  see  Zvonkin  and 

Levin  [24] . 

2.  See  Chaitin  [5]  and  [6]  and  Kolmogorov  [13] . 

3.  See  Kolmogorov  [13]  and  Solomonoff  [22] . 

4.  See  Martin-Lof  [17]. 

5.  See  Blum  [2]  for  an  abstract  definition  of  a  com- 
plexity measure. 

6.  See  Kolmogorov  [13]. 

7.  See  Martin-Lof  [17]. 

8.  See  Kolmogorov  [13]. 

9.  See  Chaitin  [7] . 

10.  See  Chaitin  [7],  pp.  5-6. 

11.  For  a  complete  discussion  of  the  correspondence 
see  Zvonkin  and  Levin  [24],  sections,  Willis  [23], 
and  Chaitin  [7] . 

12.  The  remaining  definitions  and  theorems  of  this 
section  are  all  from  Chaitin  [7] . 

13.  The  extent  to  which   P(s)  and   P(s/t)   are  indepen- 
dent of  the  particular  universal  computer   U   is 
not  explicitly  discussed.   However,  Theorem  3.7  in 
Chaitin  [7]  and  Chapter  4  in  this  paper  shed  some 
light  on  the  situation. 

14.  See  Loveland  [16]  where  this  theorem  and  its  con- 
verse are  proven. 

15.  See  Barzadin  [1],  Theorems  1  and  2. 
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Chapter  2 . 

1.  See  Feller  [11],  p.  210,  problem  5. 

2.  See  Martin-Lof  [18]  and  Zvonkin  and  Levin  [24], 
Theorem  2.6. 

3.  See  Loveland  [16] . 

4.  See  Daley  [9]  . 

5.  See  Martin-Lof  [18]. 

6.  See  Chaitin  [7] . 

7.  See  Chaitin  [5] . 

Chapter  3. 

1.   This  conjecture  was  related  in  a  conversation  with 
G.  Cha. 

Chapter  5. 

1.  See  Grzegorczyk  [12]  where  all  properties  of  the 
Grzegorczyk  hierarchy  stated  in  this  section  are 
proven. 

2.  For  a  detailed  formulation  of  a  subrecursive  formal- 
ism with  the  required  properties  see  Minsky  [20] , 
Chapter  11,  Meyer  and  Ritchie  [19],  and  Constable 
[8]. 

3.  See  Daley  [9] . 

4.  See  Blum  [2] . 

5.  See  Chaitin  [6],  Section  5. 

6.  See  Lin  and  Rado  [15]. 

7.  See  Chaitin  [6],  Section  5. 
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